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(57) Abstract: A method and system for maintaining the line of sight of an airborne camera fixed on a target by compensating 
for overflight velocity of the aircraft. The compensation system automatically commands an angular velocity of the line of sight 
to maintain the camera pointing at the target being overflown. This angular velocity of the line of sight is computed based upon 
the aircraft overflight velocity and upon a vector from the aircraft to the target. This automatic compensation for aircraft overflight 
velocity causes the line of sight to remain fixed upon the target. The compensation system drives a gimbal system upon which the 
camera is mounted to perform this compensation automatically. 
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COMPENSATION FOR OVERFLIGHT VELOCITY WHEN STABILIZING AN 

AIRBORNE CAMERA 

CROSS REFERENCE TO RELATED APPLICATIONS 

[0001] The present application claims priority to pending U.S. Provisional 

Application No. 60/440,983, filed January 17, 2003, and U.S. Non-Provisional 

Application No. , filed January 15, 2004, both incorporated herein 

in their entirety by reference. 

TECHNICAL FIELD 

[0002] The described technology relates to an apparatus for stabilization of 

the pointing direction of an airborne camera. 

BACKGROUND 

[0003] The pointing direction or line of sight of an airborne camera that is 

fixed to the body of an aircraft moves about as the aircraft maneuvers. For 
example, as the aircraft rolls, the camera rolls up or down depending on 
which side of the aircraft it is mounted. If the camera is inertially stabilized, 
rather than fixed to the body of the aircraft, its line of sight will remain in a 
fixed direction relative to the body of the aircraft as the aircraft maneuvers. 
When the camera is inertially stabilized, its field of view sweeps across a 
scene at a velocity equal to the overflight velocity of the aircraft. For 
example, if the camera is pointing directly down with its scan aligned with the 
body of the aircraft and the aircraft is traveling at 360 mph and if the camera's 
field of view is one mile, then an object entering the field of view will exit the 
field of view in 10 seconds. 

[0004] Inertial stabilization is typically effected by mounting an airborne 

camera within a gimbal system, and driving the gimbals based on the roll, 
pitch, and heading of the aircraft. Such an approach can deliver high-quality 
stabilized pointing, but aircraft overflight velocity is not removed. Operator 
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inputs are used to command changes in line of sight to compensate for 
overflight velocity. Such systems are the subject of U.S. Patent 5,897,223; 
U.S. Patent 3,638,502; U.S. Patent 4,989,466; U.S. Patent 4,643,539; and 
U.S. Patent 5,184,521. Figure 1 illustrates a prior technique that does not 
correct for overflight velocity of the aircraft over the target. In such prior art, 
the line of sight of the camera is inertially stabilized. In this example, as the 
aircraft 100 circles 101, the camera's line of sight sweeps out a circle 102. 
This stabilization sweeps the imaged area across the ground at a velocity 
equal to aircraft overflight velocity. To compensate for the overflight velocity, 
an operator would manually need to keep the camera pointing at the target, 
[ooos] Automatic compensation for aircraft overflight velocity causes the line 

of sight to remain fixed upon a selected position relative to the ground. If the 
gimbal system performs this compensation automatically, then the operator is 
relieved of this burden. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0006] Figure 1 illustrates a prior technique requiring operator correction for 

overflight velocity of the aircraft over the target 
[ooo7] Figure 2 illustrates the compensation system in which the line of sight 

of the camera is not inertially stabilized but is rather given an angular 

velocity. 

[ooos] Figure 3 illustrates the effect of this compensation upon the image 

sequence created by the camera. 
[0009] Figure 4 illustrates how the required angular velocity of the line of sight 

is calculated. 

[0010] Figure 5 is a diagram illustrating the reference frames used in the 

compensation system. 
[0011] Figure 6 is a flow diagram illustrating the processing of a component 

that establishes the position of the target in one embodiment. 
[0012] Figure 7 is a flow diagram illustrating the processing of a component 

that locks on the target in one embodiment. 
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DETAILED DESCRIPTION 

[0013] A method and system for maintaining the line of sight of an airborne 

# camera fixed on a target by compensating for overflight velocity of the aircraft 

is provided. The compensation system automatically commands an angular 
velocity of the line of sight to maintain the camera pointing at the target (e.g., 
a selected location on the ground) being overflown. This angular velocity of 
the line of sight is computed based upon the aircraft overflight velocity and 
upon a vector from the aircraft to the target. This automatic compensation for 
aircraft overflight velocity causes the line of sight to remain fixed upon the 
target. The compensation system drives a gimbal system upon which the 
camera is mounted to perform this compensation automatically. As a result, 
the camera can remain pointed at the target without needing operator 
intervention. 

[0014] In one embodiment, the compensation system calculates the radians 

per second in the scan and tilt directions that the camera needs to move to 
compensate for the overflight velocity of the aircraft. The compensation 
system periodically (e.g., 20 times per second) calculates the radians and 
adjusts the gyro's angular control of the gimbal system. The compensation 
system calculates both a static and a dynamic adjustment. The static 
adjustment represents the difference in radians between the actual line of 
sight of the camera and the needed line of sight of the camera based on the 
current positions of the aircraft and the target. The dynamic adjustment 
represents the radians per second that the camera needs to move to 
compensate for the overflight velocity. The compensation system combines 
the static and dynamic adjustments to give the actual adjustment The 
dynamic adjustments allow for smooth and continuous adjustments to be 
made based on velocity, whereas the static adjustments tend to correct for 
accumulated errors in the line of sight The compensation system may apply 
a weighting factor to the static and dynamic adjustments. For example, if the 
position of the aircraft is not very accurate, then the static adjustment may be 
given a low weight relative to the dynamic adjustment because the dynamic 
adjustment would be assumed to be more accurate. By combining the static 
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and dynamic adjustments, the compensation system can more accurately 
keep the line of sight of the camera on the target. 
[0015] Figure 2 illustrates the line of sight of a camera that is given an 

angular velocity by the compensation system. This angular velocity 
compensates for the overflight velocity of the aircraft with respect to the 
target. The center of the imaged area is held fixed upon a location on the 
ground. The aircraft 200 is traveling at a velocity represented by vector Vned. 
Initially, the line of sight of the camera is in the direction represented by 
vector 201 . As the aircraft flies at velocity VW D , the angular velocity of the 
camera is represented by vector 207. . The line of sight of the camera 
transitions through vector 201 to vector 206 in order to keep the line of sight 
on the target. 

(0016] Figure 3 illustrates the effect of the compensation upon the image 

sequence created by a camera. This compensation causes the field of view 
to remain fixed upon a selected location, rather than sweep along at aircraft 
overflight velocity. In this example, as the aircraft 300 circles 301, the 
camera's line of sight stays centered on target 302. 

tooi7] Figure 4 illustrates calculation of the angular velocity of the camera to 

keep the line of sight centered on the target. The calculation employs the 
relative velocity between the aircraft and the target, together with the vector 
from the aircraft to the target. This calculation is performed continuously, as 
the aircraft velocity and position change and as the target location changes. 
The compensation system initially acquires 401 the target position by, for 
example, an operator centering the camera on the target. The compensation 
system can calculate the position of the target based on the altitude of the 
target (e.g., sea level), aircraft altitude and position, and angles of the gimbal 
system. The compensation system then updates 402 the line of sight based 
on the current aircraft position and the target position. The compensation 
system calculates 403 the rate of change of the line of sight based on the 
current aircraft position and the target position along with relative velocity 
between aircraft and target. The update of the line of sight is the static 
adjustment, and the update of the rate of change is the dynamic adjustment. 



WO 2004/102473 PCT/US2004/00J206 

5 

The compensation system loops calculating the static and dynamic 
adjustments. The dynamic adjustment allows for the adjusting of the line of 
sight continuously, that is the gimbals are moved at the specified angular rate 
between calculations. The static adjustment allows for the line of sight to be 
corrected due to accumulated errors caused by the dynamic adjustment and 
prior static adjustments. If the dynamic adjustment were not made, the error 
between the static adjustments would be much larger, 
is] Figure 5 is a diagram illustrating the reference frames used in the 

compensation system in one embodiment. The reference frame of the earth 
is represented by north N, east E, and down D coordinates. The position of 
an aircraft, which may be provided by an altimeter and a GPS system, is in 
the earth reference frame. The reference frame of the body of the aircraft is 
represented by heading B1, pitch B2, and roll B3 coordinates that may be 
provided by the aircraft's gyros. The reference frame of the camera is 
represented by a line of sight C1 , tilt C2, and scan C3 coordinates. In one 
embodiment, the camera is controlled by an inertial stabilization system that 
controls the gimbal motors to control the orientation of the C1, C2, and C3 
camera axes. The compensation system receives camera scan and tilt rate 
information from the camera rate gyros and adjusts these rates to further 
account for the overflight velocity. The inertial stabilization system maintains 
the orientation of the camera to compensate for the maneuvering of the 
aircraft. 

)19] The compensation system inputs the altitude of the target and an 

indication of the line of sight of the camera when pointing at the target. The 
compensation system initially calculates the position of the target in the 
reference frame of the earth R B target using the position of the aircraft R E 
aircraft. The compensation system then repeatedly adjusts the gimbal angles 
so that the target is in the line of sight of the camera to compensate for the 
overflight velocity. The compensation system calculates the velocity of the 
aircraft relative to the target and adjusts the gimbal angles as the aircraft 
moves so that the target will remain in the line of sight This adjustment is 
referred to as the dynamic difference in the line of sight because it 
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compensates for movement of the aircraft relative to the target. Because the 
adjustments may not perfectly compensate for the overflight velocity, the 
compensation system also calculates the static difference between the actual 
line of sight of the camera and the line of sight needed to point to the target. 
The compensation system includes this static difference in the adjustment to 
the gimbal angles. Thus, the adjustments for dynamic differences based on 
velocity compensates for the overflight velocity/and the adjustments for static 
differences based on the line of sight of the camera compensates for 
variations between the intended and the actual line of sight. 
[0020] The compensation system uses transformation matrices to represent 

the current orientation of the body of the aircraft relative to the earth 
reference frame and the current orientation of the camera to the body 
reference frame. The camera reference frame relative to the body of the 
aircraft reference frame is represented by a transformation matrix Ccb for 
transforming a vector from the body reference frame to the camera reference 
frame. Ccb is a 3-by-3 matrix whose columns are orthogonal and normalized, 
also referred to as a matrix of direction cosines. The following equation 
represents the conversion of a position in the body reference frame to the 
camera reference frame: 

[0021] R c = C^R® 

[0022] where R B represents the position in the body reference frame and R c 
represents the position in the camera reference frame. An example Ccb is 



2-1/2 


_2' 1/2 


0 


(2) 


2"1/2 


2'1/2 


0 




0 


0 


1 





[0023] The matrix Ccb is set based on the angles of the gimbal relative to the body. 
Thus, this matrix represents the current gimbal angles. A matrix C B e is for 
transforming from the earth reference frame to the body reference frame. 
Thus, the matrix C B e represents the heading, pitch, and roll of the aircraft as 
measured by the gyro of the aircraft. 



WO 2004/102473 PCT/US2004/001206 

7 

[0024] Figure 6 is a flow diagram illustrating the processing of a component of 

the compensation system that establishes the position of the target in one 
embodiment. The component inputs the position of the aircraft an 
indication of the pitch, roll, and heading of the aircraft; an indication of the 
direction of the target in the camera reference frame; and the altitude of the 
target. The indication of the pitch, roll, and heading of the aircraft is 
represented by a transformation matrix Cue for transforming a position in the 
earth reference frame to the body reference frame. The indication of the line 
of sight of the camera (which is pointing at the target) is represented by a 
transformation matrix Ccb for transforming a position in the body reference 
frame to the camera reference frame. The direction of the target may be 
indicated by centering the camera on the target. Alternatively, an operator 
could point to the target as it is being displayed using a pointing device such 
as a mouse. The line of sight of the camera is given by vector (1,0,0) in the 
camera reference frame. Thus, if the target is centered in the camera image, 
then its coordinates are (1,0,0). Otherwise, its coordinates have non-zero 
scan and tilt values. In block 601, the component calculates a transformation 
matrix for transforming from the earth reference frame to the camera 
reference frame as follows: 

[0025] Cce ~ Ccb Cbe 

[0026] In block 602, the component calculates the line of sight of the camera in the 

earth reference frame as 
[0027] L e = Cce t (1,0,0) t (4) 

[0028] where L E is the line of sight of the camera in the earth reference frame and 
where the superscript T represents the transform of the matrix or vector. In 
block 603, the component calculates the altitude difference between the 
aircraft and the target. The altitude difference between the aircraft and the 
target is 

[0029] AD = Dairenft - Durget (^) 
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[0030] where D represents the down coordinate of the earth reference frame. In 
blocks 604-606, the component adjusts L E by AD to convert it to the actual 
position vector AR E from the aircraft to the target The adjustment multiplies 
each coordinate by AD divided by the down coordinate of the line of sight of 
the camera in the earth reference frame represented as L E (D) as follows: 

AD 



AR B (D)= L e (D)* 



L*(Z>) 



[0031] AR E (N)=L E (N)*^^ (6) 

AR E (E)= L e (E) * ■ m 



L*(D) 

[0032] where AR E (x) represents the difference in position vector between the aircraft 
and the target in the earth reference frame, L E (x) represents the line of sight 
of the camera in the earth reference frame, x represents a component of the 
reference frame, and AD represents the difference in altitude between the 
aircraft and the target. In an alternate embodiment, the actual position vector 
from the aircraft to the target AR E can be calculated by multiplying the line of 
sight of the camera L E times the range K as follows: 

[0033] AR E = L E * K (7) 

[0034] The range can be determined using a range finder instrument. In block 607, 
the component calculates the position of the target in the earth reference 
frame as 

r° 035 l . R Lget ~AR E +R E rcraft (8) 

[0036] The component then returns the position of the target . 

[0037] Figure 7 is a flow diagram illustrating the processing of a component 

that locks on the target in one embodiment. The component may be invoked 
several times per second and is passed the current position of the aircraft 
and target (Rf^ and R^); an indication of the pitch, roll, and heading of 
the aircraft (C B e); and an indication of the line of sight of the camera (Ccb). In 
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block 701 , the component calculates a transformation matrix for transforming 
from the earth reference frame to the camera reference frame as follows: 

[0038] Cce = Ccb C BE (9) 

[0039] In block 702, the component calculates the position vector (AR B ) between 
the aircraft and the target in the earth reference frame as: 

[0040] AR^RL^RLa* (10) 

[0041] In block 703, the component transforms AR E to the camera reference frame 
as follows: 

[0042] AR c = CceAR e (11) 

[0043] In block 704, the component normalizes the position vector between the 
aircraft and the target by dividing its coordinates by the distance from the 
aircraft to the target as follows: 

[0044] AR^^l (12) 

AR cl 



[0045] where Afi. is the normalized position vector and |AR C | is the distance from 

the aircraft to the target. The scan and tilt coordinates of the normalized 
position vector indicate the difference in radians between the actual line of 
sight of the camera and the line of sight needed to point to the target in the 
scan and tilt directions, which represents the static adjustment. In block 705, 
the component calculates the velocity vector (AV E ) between the aircraft and 
the target in the earth reference frame as: 

[0046] AV E =V^-VL^ (13) 

[0047] In block 706, the component transforms the velocity vector AV E to the camera 
reference frame as follows: 

[0048] AV C = C CE AV E (14) 

[0049] where AV C represents the velocity vector in the camera reference frame. In 
block 707, the component normalizes the velocity vector between the aircraft 



WO 2004/102473 PCTYUS2004/001206 

10 

and the target by dividing its coordinates by the distance from the aircraft to 
the target as 

[0050] A^ 0 -^ (15) 

AR 



[0051] where AV C is the normalized velocity vector and |AR C | is the distance. The 

scan and tilt coordinates indicate the radians per second that the camera 
needs to move to compensate for the overflight velocity in the scan and tilt 
directions. In block 708, the component calculates the adjustment for the 
signal from the inertial stabilizer to the gimbal for the scan of the camera as 

[0052] A(S) = AR C (S)*W+AV C (S) (16) 

[0053] where A(S) is the adjustment for the scan coordinate in radians per second, 
AR C (S) is the scan coordinate of the normalized position vector, AV° (S) is 

the scan coordinate of the normalized velocity vector, and W is a weighting 
factor for combining the velocity and position variation in units of per second. 
The weighting factor can be increased or decreased to compensate for the 
accuracy of the measurements used to calculate the dynamic and static 
adjustments. For example, if the attitude of the aircraft is known to be not 
very accurate, then the weighting factor may be small. Conceptually, AR C (S) 

is the error between the line of sight of the camera and where it should be to 
point to the target, and AV C (S) is the velocity at which the camera should be 

scanning to keep the target in its line of sight. The compensation system 
compares the adjustment to the scan rate of the camera provided by the gyro 
and uses the difference in rate to control the velocity of the gimbal motors. 
For example, if the gyro indicates that the camera should scan at .1 radian 
per minute and the compensation system indicates that the scan rate should 
be .01 radians per minute, then the rate provided to the gimbal system would 
be .11 radians per minute. In block 709, the component calculates the 
adjustment for the signal from the inertial stabilizer to the gimbal for the tilt of 
the camera in an analogous manner and then returns the adjustments. 
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54] In one embodiment, the compensation system calculates the altitude of 

the target based on a user indicating the position of a target on the camera 
image. For example, a user may center the target within the image at one 
time and then at a later time again center the target within the image. Thus, 
the compensation system has two lines of sight to the target from two 
different aircraft positions. The compensation system can convert these lines 
of sight to the earth reference frame and then calculate their point of 
intersection in the earth frame of reference, which gives the altitude of the 
target. The compensation system can also calculate the velocity of the target 
based on a series of centering the image on the target or a series of 
indications where the target is within the image (e.g., clicking on the target). 
This velocity and altitude can be provided to compensate for the overflight 
velocity. 

155] One skilled in the art will appreciate that although specific 

embodiments of the compensation system have been described for purposes 
of illustration, various modifications can be made without deviating from the 
spirit and scope of the invention. For example, the principles of the 
compensation system may be used on a transport mechanism other than an 
airplane, such as a satellite, a rocket, a missile, a train, an automobile, and 
so on. In addition, the compensation system may be used to control the "line 
of sight" or "orientation" of a device other than a camera, such as a laser or a 
weapons system. In such a case, a positioning scope may be used to 
establish the initial line of sight to the target. Accordingly, the invention is 
described by the appended claims. 
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CLAIMS 
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I/We claim: 

[ci] 1. A method for controlling the line of sight of a camera to remain 

fixed on a target, the camera being on a vehicle whose current position is 
moving at a velocity relative to a current position of the target, the method 
comprising: 

setting an initial line of sight for the camera that is aimed at the current 
position of the target; 

periodically setting an angular velocity for moving the line of sight of 
the camera that is calculated to compensate for the current 
velocity of the vehicle relative to the target; 

periodically setting the line of sight of the camera based on an 
adjustment angle that is calculated to compensate for a 
difference between a needed line of sight derived from the 
current position of the vehicle and the current position of the 
target and the current line of sight of the camera; and 

maintaining the line of sight of the camera in accordance with the set 
angular velocity and set line of sight as the attitude of the 
vehicle changes relative to the current position of the target 

( c2 J 2. The method of claim 1 wherein the target is at a fixed position. 

[ c3 l 3. The method of claim 1 wherein the vehicle is airborne. 

[c4] 4. The method of claim 1 wherein the initial line of sight of the 

camera is set based on an operator centering the line of sight of the camera 
on the target. 
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[c5] 5. The method of claim 4 including calculating the current position 

of the target based on the current position of the vehicle and altitude of the 
target. 

[c6] 6. The method of claim 1 wherein the initial line of sight of the 

camera is set based on the current position of the target and the current 
position and attitude of the vehicle. 



[c7] 7. The method of claim 7 wherein the attitude includes pitch, roll, 

and heading. 

[c8] 8. The method of claim 1 wherein a gyroscope is used to maintain 

the line of sight of the camera. 

[c9] 9. A method for controlling the orientation of a device to remain 

fixed on a target, the device being on a vehicle that is moving at a velocity 
relative to a current position of the target, the method comprising: 

setting an initial orientation for the device that is aimed at the current 

position of the target; 
periodically setting a velocity for moving the orientation of the device 
that is calculated to compensate for the current velocity of the 
vehicle relative to the target; 
periodically setting the orientation of the device that is calculated 
based on the current position of the device and the current 
position of the target; and 
maintaining the orientation of the device in accordance with the set 
velocity and set orientation as the attitude of the vehicle 
changes relative to the current position of the target. 

[do] 1 0. The method of claim 9 wherein the target is at a fixed position. 



[en] 1 1 . The method of claim 9 wherein the device is a camera. 
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[ci2] 12. The method of claim 9 wherein the vehicle is airborne. 

[ci3] 1 3. The method of claim 9 wherein the vehicle is land based. 

r 

[ci4] 14. The method of claim 9 wherein the initial orientation of the A 

device is set based on an operator centering the orientation of the device on 
the target. 

[d5] 15. The method of claim 14 including calculating the current 

position of the target based on the current position of the vehicle and 
difference in altitude between the current position of the vehicle and current 
position of the target. 

[d6] 16. The method of claim 9 wherein the initial orientation of the 

device is set based on the current position of the target and the current 
position and attitude of the vehicle. 

[d7] 17. The method of claim 9 wherein a gyroscope is used to maintain 

the orientation of the device as the vehicle maneuvers. 

[ci8] 18. A system for controlling the orientation of a device to remain 

fixed on a target, the device being on a vehicle that is moving relative to a 
target, comprising: 

a component that sets an initial orientation for the device so that the 

device is aimed at the target; 
a component that periodically establishes an adjustment rate for 
moving the orientation of the device during the period that is 
calculated to compensate for the velocity of the vehicle relative # 
to the target; 

a component that periodically establishes a new orientation for the # 
device that is calculated based on a current position of the 
device and a current position of the target; and 
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a component that maintains the orientation of the device in accordance 
with the established adjustment rate and established orientation 
as the vehicle maneuvers. 

[d9] 1 9. The system of claim 1 8 wherein the target is moving. 

[c20] 20. The system of claim 1 8 wherein the device is a camera. 

[c2i] 21 . The system of claim 1 8 wherein the vehicle is airborne. 

[c22] 22. The system of claim 1 8 wherein the vehicle is space based. 

[c23] 23. The system of claim 18 wherein the initial orientation of the 

device is set based on an operator centering the orientation of the device on 
the target. 

[c24] 24. The system of claim 23 including calculating an initial position of 

the target based on an initial position of the vehicle and an initial difference in 
altitude between the vehicle and the target. 

[c25] 25. The system of claim 18 wherein the initial orientation of the 

device is set based on the current position of the target and the current 
position of the vehicle. 

[c26] 26. The system of claim 18 wherein a gyroscope is used to maintain 

the orientation of the device as the vehicle maneuvers. 



[c27] 27. The system of claim 18 wherein the adjustment rate is angular 

velocity of a gimbal on which the device is mounted. 
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[c28] 28. A method for keeping a device locked on a target, the device 

being on a vehicle and being initially oriented towards the target, the method 
comprising: 

dynamically adjusting the orientation of the device based on 
calculations to compensate for a velocity of the vehicle relative 
to the target; 

statically adjusting the orientation of the device that is calculated 
based on a current position of the device and a current position 
of the target; and 

maintaining the orientation of the device in accordance with the 
dynamic and static adjustments as the vehicle maneuvers. 

[c29] 29. The method of claim 28 wherein the calculation for the dynamic 

adjustment is performed at a time interval. 

[c30] 30. The method of claim 29 wherein the dynamic adjustment is an 

adjustment rate that is applied continuously during the time interval. 

[c3i] 31. The method of claim 28 wherein the calculation for the static 

adjustment is performed at a time interval. 

[c32] 32. The method of claim 31 wherein the orientation is statically 

adjusted once a time interval. 
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